Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unzip files after retrieving from external SFTP site #74

Merged
merged 12 commits into from
Jul 5, 2024
Merged

Conversation

somesylvie
Copy link
Contributor

Unzip files after retrieving from external SFTP site

After retrieving a file from an external SFTP site, we check if it is a zip file. If no, place file into import, if yes, unzip the file. Place resulting individual files into import for processing, unlocking them with a password if appropriate. If there are errors reading any individual files within the zip, we add that info to a document and upload it to failures

We also updated test naming to use a single consistent pattern

Issue

CDCgov/trusted-intermediary#1079

Checklist

  • I have added tests to cover my changes
  • I have added logging where useful (with appropriate log level)
  • I have added GoDocs where required
  • I have updated the documentation accordingly

Note: You may remove items that are not applicable

somesylvie and others added 9 commits July 1, 2024 17:25
…initialize correctly

Co-Authored-By: pluckyswan <96704946+pluckyswan@users.noreply.github.com>
Co-Authored-By: Samuel Aquino <saquino@flexion.us>
Co-Authored-By: jcrichlake <145698165+jcrichlake@users.noreply.github.com>
Co-Authored-By: jherrflexion <118225331+jherrflexion@users.noreply.github.com>
…ad unzipped files to import; use constants for repeated strings; a lil cleanup and refactoring

Co-Authored-By: pluckyswan <96704946+pluckyswan@users.noreply.github.com>
Co-Authored-By: Samuel Aquino <saquino@flexion.us>
Co-Authored-By: jcrichlake <145698165+jcrichlake@users.noreply.github.com>
Co-Authored-By: jherrflexion <118225331+jherrflexion@users.noreply.github.com>
…it can be reused

Co-Authored-By: pluckyswan <96704946+pluckyswan@users.noreply.github.com>
Co-Authored-By: Samuel Aquino <saquino@flexion.us>
Co-Authored-By: jcrichlake <145698165+jcrichlake@users.noreply.github.com>
Co-Authored-By: jherrflexion <118225331+jherrflexion@users.noreply.github.com>
Co-Authored-By: pluckyswan <96704946+pluckyswan@users.noreply.github.com>
Co-Authored-By: pluckyswan <96704946+pluckyswan@users.noreply.github.com>
…sed on file type; update tests

Co-Authored-By: pluckyswan <96704946+pluckyswan@users.noreply.github.com>
Co-Authored-By: jcrichlake <145698165+jcrichlake@users.noreply.github.com>
Co-Authored-By: jherrflexion <118225331+jherrflexion@users.noreply.github.com>
Co-Authored-By: pluckyswan <96704946+pluckyswan@users.noreply.github.com>
Co-Authored-By: jcrichlake <145698165+jcrichlake@users.noreply.github.com>
Co-Authored-By: jherrflexion <118225331+jherrflexion@users.noreply.github.com>
Co-Authored-By: pluckyswan <96704946+pluckyswan@users.noreply.github.com>
Co-Authored-By: jcrichlake <145698165+jcrichlake@users.noreply.github.com>
Co-Authored-By: jherrflexion <118225331+jherrflexion@users.noreply.github.com>
Co-Authored-By: pluckyswan <96704946+pluckyswan@users.noreply.github.com>
Co-Authored-By: jcrichlake <145698165+jcrichlake@users.noreply.github.com>
Co-Authored-By: jherrflexion <118225331+jherrflexion@users.noreply.github.com>
Co-Authored-By: pluckyswan <96704946+pluckyswan@users.noreply.github.com>
Co-Authored-By: jcrichlake <145698165+jcrichlake@users.noreply.github.com>
Copy link
Contributor

@saquino0827 saquino0827 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A ton of refactor work in here and it looks great! Nice job all.

… some log statements

Co-Authored-By: jcrichlake <145698165+jcrichlake@users.noreply.github.com>
Co-Authored-By: pluckyswan <96704946+pluckyswan@users.noreply.github.com>
Co-Authored-By: jcrichlake <145698165+jcrichlake@users.noreply.github.com>
Co-Authored-By: pluckyswan <96704946+pluckyswan@users.noreply.github.com>
Co-Authored-By: Samuel Aquino <saquino@flexion.us>
Co-Authored-By: Tiffini Johnson <86614374+tjohnson7021@users.noreply.github.com>
Copy link

sonarcloud bot commented Jul 5, 2024

@somesylvie somesylvie merged commit baf69f5 into main Jul 5, 2024
13 checks passed
@somesylvie somesylvie deleted the unzip-files branch July 5, 2024 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants